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(57) Abstract 

After decoding of an MPEG bitstream to the form of a video signal, an information stream signal in the form of all or pan of the 
MPEG bitstream. is transmitted alongside the video signal, embedded in the least significant bit. Recording may simply involve extraction 
of the embedded MPEG bitstream or. if only part of the bitstream has been embedded, an essentially "dumb" coding operation guided by 
the information stream signal. 



3NSOOCID:<WO 980301 7A1> 



FOR THE PURPOSES OF INFORMATION ONLY 



Codes used to identify States party to the PCT on the front pages of pamphlets publishing international applications under the PCT. 



AL 


Albania 


ES 


AM 


Armenia 


FI 


AT 


Austria 


FR 


AU 


Australia 


GA 


AZ 


Azerbaijan 


CB 


BA 


Bosnia and Herzegovina 


CE 


BB 


Barbados 


GH 


BE 


Belgium 


CN 


BF 


Burkina Faso 


GR 


BG 


Bulgaria 


HU 


BJ 


Benin 


IE 


BR 


Brazil 


IL 


BY 


Belarus 


IS 


CA 


Canada 


IT 


CF 


Central African Republic 


JP 


CC 


Congo 


KE 


CH 


Switzerland 


KG 


CI 


Cote d'lvoire 


KP 


CM 


Cameroon 




CN 


China 


KR 


CU 


Cuba 


KZ 


CZ 


Czech Republic 


LC 


DE 


Germany 


LI 


DK 


Denmark 


LK 


EE 


Estonia 


LR 



Spain 
Finland 
France 
Gabon 

United Kingdom 

Georgia 

Ghana 

Guinea 

Greece 

Hungary 

Ireland 

Israel 

Iceland 

Italy 

Japan 

Kenya 

Kytgyzstan 

Democratic People's 

Republic of Korea 

Republic of Korea 

Kazakstan 

Saint Lucia 

Liechtenstein 

Sri Lanka 

Liberia 



LS Lesotho 

LT Lithuania 

LU Luxembourg 

LV Latvia 

MC Monaco 

MD Republic of Moldova 

MG Madagascar 

MK The former Yugoslav 
Republic of Macedonia 

ML Mali 

MN Mongolia 

MR Mauritania 

MW Malawi 

MX Mexico 

NE Niger 

NL Netherlands 

NO Norway 

NZ New Zealand 

PI- Poland 

PT Portugal 

HO Romania 

KU Russian Federation 

SD Sudan 

SE Sweden 

SG Singapore 



SI 


Slovenia 


SK 


Slovakia 


SN 


Senegal 


sz 


Swaziland 


TD 


Chad 


TC 


Togo 


TJ 


Tajikistan 


TM 


Turkmenistan 


TR 


Turkey 


TT 


Trinidad and Tobago 


UA 


Ukraine 


UG 


Uganda 


US 


United Slates of America 


UZ 


Uzbekistan 


VN 


Viet Nam 


YU 


Yugoslavia 


ZW 


Zimbabwe 



WO 98/03017 





PCT/GB97/01862 



- 1 - 



10 



15 



20 



25 



VIDEO SIGNAL COMPRESSION 

The invention relates to video signal compression. 

In an important example, the invention concerns the MPEG-2 video 
signal compression standard, ISO/IEC 13818-2, though it can be applied to 
any video compression system that is liable to degradation when coding and 
decoding are cascaded. 

There has already been disclosed (WO-A-9535628) the use of a 
signal which accompanies an MPEG bitstream and which carries information 
about the bitstream for use in a downstream process, for example, the re- 
encoding of a decoded MPEG picture. This signal is provided in parallel 
and is sent along an appropriate side channel to accompany a 
decompressed signal from a compression decoder to a subsequent encoder. 

Where equipment has been specifically designed for use with such a 
signal, considerable advantage can be gained and many of the problems 
previously associated with cascaded coding and decoding processes are 
removed or ameliorated by using in a downstream coding process, key 
information concerning upstream coding and decoding. 

It is an object of the present invention to extend these advantages, in 
part or in whole, to arrangements which include equipment not specifically 
designed for use with such a signal. 

Accordingly, the present invention consists in one aspect in a video 
signal process wherein a compression encoded signal is compression 
decoded for passage along a video pathway, characterised in that the 
compression encoded signal or a portion thereof, accompanies the video 
signal along the video pathway. 

Suitably, said portion of the compression encoded signal comprises a 
compressed form of the information bus as detailed in the above mentioned 
WO-A-9535628. This Information Bus may include the following 
information: picture dimensions; frame rate; picture structure (frame-coded 
or field-coded); picture type (l,P or B); whether macroblocks are intra-coded 
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or use prediction; whether forward, backward or bi-directional prediction is 
used; motion vectors; quantiser visibility weighting matrices; quantiser step 
and buffer state of a downstream decoder. 

In a further aspect, the present invention consists in the transport 
along a video pathway of a signal, containing all or part of an MPEG 
bitstream, alongside a video signal decoded from the MPEG bitstream. 
In one form of the invention, said signal is MPEG compliant. 
The advantage of such an arrangement is that bitstream information 
can pass transparently through studio processing equipment to facilitate re- 
encoding of the output to MPEG format without impairments due to 
cascading. For the purposes of this specification, the new signal is called 
the Information Stream. 

The invention will now be described by way of example with reference 
to the accompanying drawings, in which:- 

Figure 1 is a block diagram of a studio installation that makes use of 
the Information Stream; 

Figure 2 is a block diagram of an Information Stream coder; and 
Figure 3 is a block diagram of an Information Stream decoder. 
Referring initially to Figure 1, the installation has MPEG bitstreams at 
its input and output but uses 'conventional' studio processing equipment 
working on uncompressed video signals. This block diagram can be viewed 
in two parts: the main signal path that runs along the top of the diagram, and 
the Information Stream path that runs along the bottom. 

In the main signal path, an MPEG bitstream enters an MPEG decoder 
front-end 10 which produces DCT coefficients and an Information Bus. 
These are processed further by the MPEG decoder display processing block 
12 to produce an ITU-R Rec. 656 video signal for the studio equipment 
shown at 14. This equipment may comprise, for example, a vision mixer, a 
simple switch or a DVE machine. The output of the studio equipment 14 is 
analysed by an MPEG coder front-end 16 which passes a suitably formatted 
picture signal and an Information Bus to an MPEG coder 18. As it stands, 
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this configuration necessitates a full MPEG re-coding operation at the output 
of the studio equipment, an operation which will introduce impairments due 
to cascading. 

The problem of impairments due to cascading is overcome by the 
5 addition of the Information Stream path, the various stages of which are 
provided by the present invention. The DCT coefficients and Information 
Bus are processed in an Information Stream coder 20, which produces one 
of a variety of Information Stream signals, as will be described in more detail 
beiow. Formatting of the Information Stream is performed by an Information 

10 Stream formatter 22 and the resulting bitstream is encoded by an 

Information Stream channel adapter 24 so that it can pass through the 
studio equipment 14 together with the decoded video signal. At the output 
of the studio equipment, the Information Stream is decoded by an 
information Stream decoder 26, typically to an Information Bus that is 

15 compatible and synchronous with the Information Bus produced by the 
MPEG coder front-end 16. An Information Bus switch 28 then selects 
between the newly calculated information Bus and the Information Bus 
decoded from the Information Stream. The selected Information Bus is then 
used as the basis for re-encoding to an MPEG bitstream. 

20 The MPEG coder front-end 16, the Information Bus switch 28 and the 

MPEG dumb coder 18 together constitute an Information Stream-based 
MPEG coder. Note that the bit-rate of the MPEG bitstream at the output of 
this coder is not constrained to be the same as that of the original MPEG 
signal. 

25 Typically, the Information Stream path is used whenever the studio 

equipment has passed the Information Stream transparently. This would 
happen, for example, when the slider switch in a vision mixer is set to select 
one or other of the inputs in full. At other times, for example, when the 
slider is in an intermediate position, the Information Stream passing through 

30 the studio equipment is corrupted and cannot be used, and the picture 

information is such that new MPEG coding decisions need to be made in 
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any case. 

Turning now to Figure 2, there is illustrated an Information Stream 

coder. 

The DCT coefficients and the Information Bus are received, 
respectively by a coefficient pre-processor 30 and an Information Bus pre- 
processor 32. The two pre-processor outputs are taken to a variable length 
coder 34, the output of which passes through a buffer 36 to a time stamp 
allocation and packetisation unit 38 to produce the Information Stream 
output. The buffer 36 receives an external read control signal. The 
occupancy of the buffer 36 is monitored by a buffer control unit 40 which 
serves to control the pre-processors 30 and 32. 

The Information Stream coder is very much like the back end of an 
MPEG coder. The main difference is that the buffer control may be used in 
different ways to limit the bit-rate of the Information Stream. Reading from 
the buffer may also - as mentioned - be controlled externally. 

Examples of signals that could constitute an Information Stream are: 

i) The MPEG bitstream itself, either in the form of an 
Elementary Stream or a Packetised Elementary Stream. In this case, the 
MPEG decoder front end and the Information Stream coder would be 
bypassed and the input bitstream used directly, possibly via a delay. 

ii) The MPEG bitstream modified by removing or altering 
DCT coefficients in order to limit the bit-rate to the capacity of the chosen 
transport method, by means of the coefficient pre-processor as shown in 
Figure 2. 

iii) The MPEG bitstream modified by removing all the DCT 
coefficients. This is, in effect, a compressed version of the Information Bus, 
using the MPEG syntax as the compression method. If the bit rate available 
for the Information Stream is very low, it may be necessary in extreme 
circumstances to remove less essential elements from the bitstream by 
means of the Information Bus pre-processor as shown in Figure 2. 

iv) The Information Bus itself. 
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There are various possibilities for the format of an Information Stream 
signal, according to its timing relationship with the video signal it 
accompanies. Formatting is carried out by the Information Stream formatter 
22 of Figure 1 . Examples of possible formats for the Information Stream 
5 signal are as follows: 

i) A fixed-bit-rate signal but containing a variable number 
of bits per picture and transmitted with no regard for synchronisation to the 
video signal. In practice, the Information Stream could have a variable bit- 
rate but could be made to occupy a fixed-bit-rate channel by the use of 

10 stuffing bits 

ii) A fixed or variable-bit-rate signal which is re-ordered 
(from bitstream order to display order within the GOP structure) and time- 
shifted so that the Information Stream for each picture is co-timed with the 
video signal for that picture. 

15 iii) A mixture of the two, in that the Information Stream itself 

is asynchronous but a small slot is reserved for some picture-locked data; 
this would carry, for example, duplicates of time_code and picture_type. 

iv) A fixed-bit-rate signal which is re-ordered and time- 
shifted as described in the second option above, but additionally arranged so. 

20 that the macro-rate information for each macroblock is co-timed with the 
video signal corresponding to the macroblock. 

A number of ways have been identified in which the Information 
Stream might be transported along with a digital video signal. The 
Information Stream channel adapter of Figure 1 is responsible for this task. 

25 Examples are as follows:- 

i) In the least significant bit of the colour-difference part of 
a 10-bit ITU-R Rec. 656 signal, within the active video region only. This 
provides a raw bit-rate of 10.368 Mbit/s for the Information Stream. Care 
will be taken to ensure that the presence of the Information Stream does not 

30 cause visible impairments to the video signal and that studio equipment 

quoted as '10 bits' is indeed transparent to all ten bits of the signal when no 
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mixing or other processing is being performed. This option for the transport 
of the Information Stream is thought to be particularly ingenious. In other 
implementations, the Information Stream might be transported in the 9th or 
8th colour-difference bit, in the 10th, 9th or 8th luminance bit or in any 
5 combination of the above. Use of the 8th bit would also be appropriate for 
systems using earlier versions of the Rec. 656 standard where only 8-bit 
representation is available. 

ii) An extension of the above approach, in which any part 
of the digital video signal (not just the least significant bit) is modified by 
10 adding Information Stream data in such a way that a downstream MPEG 
coder would be unaffected. 

Hi) In the ancillary data channel carried in the blanking 
periods of the Rec. 656 signal. It would be necessary to ensure that studio 
equipment passed this information unchanged when no mixing or other 
15 processing was being performed 

iv) As an AES/EBU digital audio channel. This would be 
passed through a spare channel in the audio path of the studio equipment. 
It would be necessary to ensure that switching of that particular audio 
channel would be performed aiong with the video switching, even though the 
20 main audio channel(s) might be switched independently of the video. 

In addition to forming an appropriate transport method for the 
Information Stream, the Information stream channel adapter can optionally 
perform any or all of the following three further operations: 

i) Scrambling of the Information Stream, for example by 
modulo-2 addition to a known pseudo random binary sequence, or by using 
a self-synchronising scrambler. This may be necessary to limit the visibility 
of the Information Stream if it is carried in the least significant bits of a 
video signal. A seed for the pseudo random sequence, or even the 
sequence itself, may be transmitted as part of the Information. Stream. 
30 "> Error correction of the Information Stream using any 

known algorithm. This could include repetition of the picture-rate 
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information in several places in the picture so that spatially limited 
processing, such as caption insertion, would not cause loss of that 
information. 

iii) Modifying the video signal itself by a known noise 
5 shaping algorithm such as dynamic rounding or error feedback, to reduce 

further any residual visibility of the Information Stream in the case where it is 
carried in the video signal. For example, if the Information Stream is carried 
in the 8th luminance bit, the error incurred in replacing the original 8th 
luminance bit by the Information Stream bit can be delayed by one clock 

10 period - or passed through some more complex noise shaping network - 
and added back into the signal. A full description of the error feedback 
process is given in, for example, Mitra & Kaiser, Handbook for Digital Signal 
Processing, Wiley, 1993, p.421. 

There is shown in Figure 3, an Information Stream decoder which 

15 performs the inverse of the channel adaptation, formatting and coding 

functions to produce an Information Bus, and which provides an indication to 
the Information Bus switch as to the validity or otherwise of the Information 
Stream. In more detail, the information Stream decoder comprises a 
channel receiver 50, which passes a signal to, in turn, a descrambler 52 and 

20 an error corrector and detector unit 54. The descrambled and error 

corrected signal passes through buffer 56 to a variable length decoder 58 
which outputs coefficient information and the Information Bus. A validity 
logic unit 60 receives inputs from the error corrector and detector unit 54 
and the buffer 56 and provides an "Information Stream Valid" flag. This 

25 controls the Information Bus switch 28. 

The Information Stream will be invalid in the case of: 

(i) an error in the received Information Stream signal (detected by 
the error corrector and detector unit 54), which will indicate -for example - 
that a DVE is affecting the video signal and that the Information Bus 

30 calculated by the new MPEG coder front end must be used; 

(ii) unavailability of the decoded Information Stream signal at the 
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required time, which indicates that the Information Stream has been lost in 
the DVE, even if the video to which it refers has been passed untouched 
(this can occur if the Information Stream and the video signal are not co- 
timed); 

5 (iii) in the case of an Information Stream which is co-timed with 

the video signal, the fact that the Information Stream appears to be valid but 
contains motion vectors which point to macro-blocks in reference pictures 
which are unavailable. 

In some implementations, it is the task of the Information Stream 
10 decoder to ensure that the information Bus at its output is co-timed with the 
Information Bus at the output of the MPEG coder front end. This can be 
done by making use of the fact that the Information Stream is an MPEG- 
compliant signal which requires a buffer and conventional MPEG decoding 
technology to decode it The buffer can be used as a delay device, 
15 controlled by an external read signal, to ensure that the two Information 
Buses are synchronous. 

The Information Bus switch 28 selects between the Information Bus 
decoded from the Information Stream and the newly calculated Information 
Bus from the coder front-end. In its simplest form, it uses the validity signal 
20 generated by the Information Stream decoder to control the switch. More 
complicated options are possible, for example, selection between the two 
information Buses on a spatial basis according to the presence or absence 
of a caption inserted by the DVE. This is particularly appropriate if the 
Information Stream has been formatted in such a way as to co-time it with 
25 the picture signal on a macroblock basis. 

In the specific description so far, the decoded Information Stream has 
been assumed to be an Information Bus, providing coding decisions to the 
dumb coder whenever it is vaiid. However, as already mentioned, the 
Information Stream could in fact contain the whole bitstream, or at least 
some information about DCT coefficients as well, as the coding decisions. If 
this is the case, the dumb coder would contain some additional processing 
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to make use of the coefficient information. This might be beneficial, for 
example in the case of the dumb coder producing a bit-rate that is different 
from that of the original MPEG bitstream. 

The Information Stream can be used advantageously whenever there 
5 is a need to pass a decoded MPEG signal through any processing that is 
ultimately followed by a subsequent re-encoding operation. Figure 1 has 
already shown a typical application. It should be noted that the MPEG 
decoder shown on the left of Figure 1 could be a conventional decoder into 
which the MPEG bitstream is 'pushed' from a network or transmission 
10 channel, or it could be a decoder which 'pulls' information from a network or 
storage medium under its own timing control, as in an MPEG-based server 
station. 

Within a studio installation, it is not always necessary to re-create a 
coded MPEG signal between processes that work on the decoded digital 

15 video signal. It is sufficient to pass the signal together with the Information 
Stream through the studio and only to re-encode to MPEG at the final studio 
output, using an Information Stream based coder as described above. 
Wherever the Information Stream has passed through all the studio 
processing unscathed, it will provide the Information Bus for the final 

20 encoding operation. Otherwise, the final encoder ignores the Information 
Stream and codes the picture from scratch. 

Note that the present invention and the Information Stream concept 
are not confined to MPEG compression. It could be used with any 
compression technique, or even with a mixture, although in this case the 

25 processing of the decoded Information Stream would be significantly more 
complicated, as it would involve the re-interpretation of coding mode 
information for a different compression scheme. 

It should be understood that this invention has been described by way 
of examples only and a variety of further modifications are possible without 

30 departing from the scope of the invention. 
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CLA1MS 

1 . A video signal process wherein a compression encoded signal is 
compression decoded for passage along a video pathway, characterised in 

5 that the compression encoded signal or a portion thereof, accompanies the 
video signal along the video pathway, 

2. A video signal process characterised by the transport along a video 
pathway of a signal, containing all or part of an MPEG or other compression 

) coded bitstream, alongside a video signal decoded from said bitstream. 

3. A process according to Claim 2, wherein said signal is MPEG 
compliant. 



15 4. A process according to Claim 2. wherein said signal is the MPEG 

bitstream itself, either in the form of an Elementary Stream or a Packetised 
Elementary Stream. 

5. A process according to Claim 2, wherein said signal is the MPEG 
20 bitstream modified by removing or altering DCT coefficients in order to limit 

the bit-rate to the capacity of the chosen transport method. 

6. A process according to Claim 2, wherein said signal is the MPEG 
bitstream modified by removing all the DCT coefficients. 



25 



7. A process according to Claim 2, wherein said signal is an Information 
Bus as defined in WO-A-9535628, 



30 



8. A process according to any one of Claims 2 to 7, wherein said signal 
is a fixed-bit-rate signal containing a variable number of bits per picture and 
transmitted with no regard for synchronisation to the video signal. 
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9. A process according to any one of Claims 2 to 7, wherein said signal 
is a fixed or variable-bit-rate signal which is re-ordered (from bitstream 
order to display order within the GOP structure) and time-shifted so that the 
said signal for each picture is co-timed with the video signal for that picture. 

5 

10. A process according to Claim 9, wherein said signal is re-ordered 
and time-shifted so that the macro-rate information for each macroblock is 
co- timed with the video signal corresponding to the macroblock. 

10 11. A process according to any one of the preceding claims, wherein said 
signal is transported alongside said video signal by encoding in a less 
significant digital element of said video signal. 

12. A process according to Claim 11 , wherein said digital element 

15 comprises the least significant bit of the colour-difference part of a 10-bit 
video signal, within the active video region only. • 

13. A process according to Claim 11, wherein said digital element 
comprises the 9th or 8th colour-difference bit, the 10th, 9th or 8th luminance 

20 bit or in any combination thereof. 

14. A process according to any one of Claims 1 to 10, wherein said signal 
is transported alongside said video signal in an ancillary data channel carried 
in the blanking periods of the video signal. 

25 

15. A process according to any one of Claims 1 to 10, wherein said signal 
is transported alongside said video signal as a digital audio channel. 

16. A process according to any one of the Claims 11 to 13, wherein said 
30 signal is processed to reduce visibility in the video signal. 



3NSOOC1D: <WO 980301 7A1> 



J 





WO 98/03017 



PCT/GB97/01862 



- 12 - 



17. A process according to Claim 16, wherein said signal is scrambled. 

18. A process according to Claim 17, wherein said scrambling comprises 
addition of a known pseudo random sequence. 



19. A process according to Claim 18, wherein a seed for the pseudo 
random sequence or the sequence itself is transmitted with said signal. 

20. A process according to Claim 17, wherein said scrambling comprises 
10 the use of a self-synchronising scrambler. 

21. A process according to Claim 16, wherein said signal is by a noise 
shaping algorithm such as dynamic rounding or error feedback 



5 



15 



22. A process according to any one of the preceding claims, wherein 
said signal is error corrected. 
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